Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[opt](nereids) optimize rewrite of synchronize materialize view #45748

Merged
merged 1 commit into from
Dec 23, 2024
Merged

[opt](nereids) optimize rewrite of synchronize materialize view #45748

merged 1 commit into from
Dec 23, 2024
+151 −62

Conversation

924060929
Copy link
Contributor

@924060929 924060929 commented Dec 20, 2024 •
edited
Loading

What problem does this PR solve?

optimize rewrite of synchronize materialize view

  1. cache toSql
  2. fast parse UnboundSlot in NereidsParser.parseExpression

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

  • Behavior changed:

    • No.
    • Yes.

  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@hello-stephen
Copy link
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@924060929
Copy link
Contributor Author

run buildall

@924060929
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 39779 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 3cf737e4a9c6e83d876a84730058c9cda5815ad6, data reload: false

------ Round 1 ----------------------------------
q1	17627	7518	7321	7321
q2	2050	178	177	177
q3	10638	1114	1274	1114
q4	10558	830	803	803
q5	7633	2776	2680	2680
q6	239	148	145	145
q7	1011	623	605	605
q8	9248	1908	1923	1908
q9	6677	6461	6411	6411
q10	7050	2325	2312	2312
q11	466	244	261	244
q12	423	222	226	222
q13	17778	2891	2928	2891
q14	248	216	222	216
q15	562	504	488	488
q16	670	587	593	587
q17	994	497	587	497
q18	7318	6639	6582	6582
q19	1344	1024	1018	1018
q20	454	183	179	179
q21	3998	3122	3066	3066
q22	373	313	316	313
Total cold run time: 107359 ms
Total hot run time: 39779 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7257	7234	7236	7234
q2	338	235	227	227
q3	2945	2793	2970	2793
q4	2154	1867	1854	1854
q5	5675	5694	5706	5694
q6	222	143	152	143
q7	2272	1867	1821	1821
q8	3425	3552	3547	3547
q9	8829	9034	9039	9034
q10	3610	3532	3541	3532
q11	618	520	503	503
q12	838	626	621	621
q13	14087	3229	3042	3042
q14	322	281	296	281
q15	554	517	505	505
q16	720	666	651	651
q17	1862	1622	1624	1622
q18	8358	7832	7834	7832
q19	1738	1564	1607	1564
q20	2091	1932	1906	1906
q21	5544	5470	5430	5430
q22	653	569	570	569
Total cold run time: 74112 ms
Total hot run time: 60405 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 197103 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 3cf737e4a9c6e83d876a84730058c9cda5815ad6, data reload: false

query1	1309	988	917	917
query2	6282	2228	2238	2228
query3	10976	4665	4788	4665
query4	33046	23449	23507	23449
query5	4066	466	466	466
query6	286	185	193	185
query7	3992	301	302	301
query8	305	250	242	242
query9	9088	2715	2711	2711
query10	467	253	250	250
query11	17782	15085	15094	15085
query12	160	104	105	104
query13	1584	421	434	421
query14	10201	7173	7216	7173
query15	293	203	195	195
query16	7487	459	474	459
query17	1652	599	571	571
query18	2008	318	319	318
query19	370	176	161	161
query20	115	117	109	109
query21	206	112	106	106
query22	4924	4635	4485	4485
query23	34557	33936	33787	33787
query24	10366	2544	2548	2544
query25	635	408	411	408
query26	1017	158	153	153
query27	2351	343	336	336
query28	7705	2482	2458	2458
query29	757	414	420	414
query30	218	146	147	146
query31	1044	851	839	839
query32	113	58	57	57
query33	786	301	291	291
query34	1099	526	526	526
query35	958	780	811	780
query36	1151	959	985	959
query37	140	82	80	80
query38	4215	4356	4452	4356
query39	1532	1459	1453	1453
query40	215	106	106	106
query41	49	51	47	47
query42	118	108	105	105
query43	587	489	494	489
query44	1305	841	828	828
query45	186	172	171	171
query46	1195	730	709	709
query47	2080	1914	1953	1914
query48	435	327	333	327
query49	910	405	397	397
query50	826	395	391	391
query51	7300	7028	7163	7028
query52	103	101	90	90
query53	260	190	183	183
query54	1144	410	407	407
query55	99	76	84	76
query56	257	242	251	242
query57	1327	1138	1179	1138
query58	240	225	223	223
query59	3304	3271	3185	3185
query60	271	257	290	257
query61	115	112	108	108
query62	904	672	709	672
query63	217	193	187	187
query64	4050	685	699	685
query65	3259	3320	3275	3275
query66	794	316	297	297
query67	16639	15654	15466	15466
query68	6037	547	559	547
query69	511	251	250	250
query70	1201	1161	1131	1131
query71	501	254	253	253
query72	6463	4172	4097	4097
query73	791	361	363	361
query74	10096	8915	8897	8897
query75	3832	2621	2637	2621
query76	3985	1134	980	980
query77	614	287	263	263
query78	10221	9519	9549	9519
query79	1129	609	593	593
query80	960	441	425	425
query81	526	250	233	233
query82	281	135	118	118
query83	195	152	152	152
query84	289	70	68	68
query85	1016	306	306	306
query86	355	299	292	292
query87	4731	4627	4504	4504
query88	4057	2214	2197	2197
query89	420	299	300	299
query90	2126	187	182	182
query91	141	106	104	104
query92	66	51	52	51
query93	2148	533	536	533
query94	942	281	284	281
query95	350	254	245	245
query96	633	275	282	275
query97	2875	2668	2686	2668
query98	223	195	188	188
query99	1617	1321	1279	1279
Total cold run time: 302764 ms
Total hot run time: 197103 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 33.3 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 3cf737e4a9c6e83d876a84730058c9cda5815ad6, data reload: false

query1	0.04	0.04	0.03
query2	0.07	0.03	0.03
query3	0.24	0.07	0.08
query4	1.59	0.10	0.11
query5	0.42	0.41	0.42
query6	1.17	0.66	0.65
query7	0.02	0.01	0.02
query8	0.04	0.03	0.03
query9	0.57	0.51	0.52
query10	0.56	0.57	0.55
query11	0.15	0.10	0.11
query12	0.14	0.11	0.11
query13	0.61	0.61	0.60
query14	2.84	2.84	2.75
query15	0.90	0.83	0.82
query16	0.40	0.39	0.39
query17	1.05	1.05	1.05
query18	0.23	0.21	0.21
query19	1.97	2.00	1.83
query20	0.02	0.01	0.01
query21	15.37	0.61	0.59
query22	2.35	2.19	2.41
query23	16.93	1.10	0.79
query24	3.16	1.17	1.79
query25	0.25	0.21	0.10
query26	0.51	0.13	0.12
query27	0.04	0.04	0.04
query28	9.86	1.10	1.07
query29	12.57	3.31	3.30
query30	0.24	0.06	0.06
query31	2.87	0.38	0.38
query32	3.24	0.47	0.46
query33	3.09	3.14	3.07
query34	16.94	4.45	4.56
query35	4.55	4.45	4.46
query36	0.68	0.48	0.49
query37	0.09	0.06	0.06
query38	0.04	0.04	0.04
query39	0.04	0.03	0.03
query40	0.17	0.12	0.12
query41	0.08	0.02	0.02
query42	0.03	0.02	0.02
query43	0.03	0.03	0.04
Total cold run time: 106.16 s
Total hot run time: 33.3 s

@924060929
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 39448 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 61c18a6c5b29f8cc192cf26477d621e0dd8557d9, data reload: false

------ Round 1 ----------------------------------
q1	17575	7423	7224	7224
q2	2050	185	172	172
q3	10598	1085	1174	1085
q4	10568	704	690	690
q5	7600	2713	2712	2712
q6	243	147	145	145
q7	977	607	608	607
q8	9259	1819	1921	1819
q9	6574	6405	6426	6405
q10	7034	2277	2326	2277
q11	460	262	255	255
q12	426	224	223	223
q13	17776	2924	2939	2924
q14	259	216	208	208
q15	566	509	495	495
q16	670	611	589	589
q17	992	519	529	519
q18	7251	6806	6547	6547
q19	1384	1029	922	922
q20	453	188	184	184
q21	4178	3140	3125	3125
q22	390	322	321	321
Total cold run time: 107283 ms
Total hot run time: 39448 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7221	7197	7218	7197
q2	328	236	228	228
q3	2884	2773	2901	2773
q4	2047	1795	1782	1782
q5	5740	5606	5612	5606
q6	230	151	150	150
q7	2246	1820	1834	1820
q8	3354	3523	3491	3491
q9	8984	8986	9010	8986
q10	3564	3575	3512	3512
q11	595	501	511	501
q12	840	626	605	605
q13	13044	3144	3086	3086
q14	302	265	281	265
q15	558	518	523	518
q16	702	629	660	629
q17	1860	1635	1605	1605
q18	8203	7833	7638	7638
q19	1747	1574	1494	1494
q20	2044	1896	1975	1896
q21	5619	5351	5410	5351
q22	660	622	574	574
Total cold run time: 72772 ms
Total hot run time: 59707 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 196973 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 61c18a6c5b29f8cc192cf26477d621e0dd8557d9, data reload: false

query1	1320	934	946	934
query2	6254	2446	2264	2264
query3	10964	4886	4826	4826
query4	33117	23353	23361	23353
query5	4167	468	474	468
query6	296	202	185	185
query7	4023	317	308	308
query8	307	251	242	242
query9	9637	2714	2711	2711
query10	472	277	250	250
query11	17830	15081	15248	15081
query12	155	106	106	106
query13	1567	424	405	405
query14	9803	7810	7034	7034
query15	275	196	195	195
query16	8157	407	458	407
query17	1746	586	635	586
query18	2226	325	328	325
query19	397	163	165	163
query20	131	121	123	121
query21	206	110	115	110
query22	4610	4555	4643	4555
query23	34730	34252	33918	33918
query24	10547	2632	2652	2632
query25	633	414	410	410
query26	1222	164	160	160
query27	2355	349	334	334
query28	7498	2480	2497	2480
query29	882	419	438	419
query30	238	160	148	148
query31	1048	860	814	814
query32	98	59	58	58
query33	764	292	304	292
query34	960	555	539	539
query35	930	812	773	773
query36	1081	946	964	946
query37	137	72	75	72
query38	4269	4321	4067	4067
query39	1495	1487	1473	1473
query40	222	111	109	109
query41	51	44	43	43
query42	116	133	104	104
query43	554	510	493	493
query44	1340	849	832	832
query45	204	177	169	169
query46	1198	755	753	753
query47	2011	1924	1912	1912
query48	438	375	327	327
query49	959	381	389	381
query50	844	418	404	404
query51	7205	6993	7326	6993
query52	101	97	95	95
query53	273	193	196	193
query54	1281	424	452	424
query55	85	80	78	78
query56	255	247	250	247
query57	1293	1192	1180	1180
query58	244	225	224	224
query59	3378	3124	3115	3115
query60	292	257	250	250
query61	108	107	106	106
query62	866	669	676	669
query63	225	202	195	195
query64	3878	676	648	648
query65	3286	3206	3225	3206
query66	833	301	309	301
query67	15755	15519	15435	15435
query68	5570	555	560	555
query69	497	271	254	254
query70	1224	1075	1174	1075
query71	500	265	250	250
query72	6583	4186	4139	4139
query73	803	369	368	368
query74	10139	8915	8901	8901
query75	3768	2669	2682	2669
query76	3867	1171	1189	1171
query77	588	287	278	278
query78	10210	9411	9269	9269
query79	1764	612	603	603
query80	686	451	444	444
query81	497	237	234	234
query82	737	121	123	121
query83	164	148	152	148
query84	235	74	74	74
query85	1437	320	313	313
query86	456	310	312	310
query87	4639	4348	4429	4348
query88	3584	2231	2207	2207
query89	439	295	292	292
query90	2072	196	194	194
query91	151	106	113	106
query92	64	55	54	54
query93	2252	547	553	547
query94	876	290	285	285
query95	355	258	252	252
query96	632	283	294	283
query97	2862	2698	2674	2674
query98	255	192	196	192
query99	1576	1334	1313	1313
Total cold run time: 303324 ms
Total hot run time: 196973 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 32.3 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 61c18a6c5b29f8cc192cf26477d621e0dd8557d9, data reload: false

query1	0.04	0.03	0.05
query2	0.07	0.03	0.03
query3	0.24	0.08	0.06
query4	1.61	0.10	0.11
query5	0.42	0.41	0.42
query6	1.16	0.64	0.65
query7	0.03	0.02	0.02
query8	0.04	0.04	0.03
query9	0.58	0.50	0.51
query10	0.55	0.56	0.57
query11	0.15	0.11	0.11
query12	0.15	0.11	0.11
query13	0.60	0.63	0.60
query14	2.71	2.72	2.74
query15	0.89	0.82	0.82
query16	0.38	0.36	0.39
query17	1.07	1.08	1.00
query18	0.23	0.20	0.21
query19	1.97	1.87	2.01
query20	0.01	0.01	0.01
query21	15.38	0.56	0.58
query22	3.00	2.76	1.78
query23	17.08	0.83	0.81
query24	3.05	0.76	1.61
query25	0.29	0.19	0.07
query26	0.39	0.13	0.14
query27	0.05	0.04	0.04
query28	10.46	1.09	1.07
query29	12.57	3.21	3.21
query30	0.26	0.07	0.06
query31	2.86	0.39	0.38
query32	3.25	0.46	0.46
query33	3.06	3.12	3.10
query34	17.16	4.43	4.49
query35	4.49	4.50	4.48
query36	0.69	0.47	0.48
query37	0.10	0.06	0.05
query38	0.05	0.04	0.04
query39	0.03	0.02	0.02
query40	0.17	0.13	0.13
query41	0.07	0.02	0.02
query42	0.03	0.02	0.03
query43	0.04	0.03	0.03
Total cold run time: 107.43 s
Total hot run time: 32.3 s

@924060929
Copy link
Contributor Author

run buildall

@924060929 924060929 mentioned this pull request Dec 20, 2024
Merged
@doris-robot
Copy link

TPC-H: Total hot run time: 40144 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit cc4607c658c00e6263e71cdfcb4e459c55cfa264, data reload: false

------ Round 1 ----------------------------------
q1	17560	7363	7253	7253
q2	2043	175	175	175
q3	10558	1066	1094	1066
q4	10527	738	685	685
q5	7629	2699	2755	2699
q6	240	146	146	146
q7	979	623	612	612
q8	9290	1926	1913	1913
q9	6718	6538	6533	6533
q10	7051	2299	2296	2296
q11	459	255	257	255
q12	428	234	224	224
q13	17760	2972	2947	2947
q14	243	219	209	209
q15	567	517	503	503
q16	668	595	596	595
q17	976	523	579	523
q18	7282	6675	6662	6662
q19	1356	1071	1135	1071
q20	464	185	183	183
q21	4235	3277	3276	3276
q22	382	318	320	318
Total cold run time: 107415 ms
Total hot run time: 40144 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7181	7224	7159	7159
q2	325	232	231	231
q3	2875	2805	3049	2805
q4	1982	1781	1760	1760
q5	5364	5518	5532	5518
q6	225	139	140	139
q7	2164	1718	1678	1678
q8	3249	3404	3371	3371
q9	8561	8538	8554	8538
q10	3468	3441	3418	3418
q11	587	490	492	490
q12	790	561	609	561
q13	12841	3071	2976	2976
q14	291	263	256	256
q15	553	494	487	487
q16	661	613	616	613
q17	1802	1593	1549	1549
q18	7765	7367	7566	7367
q19	1693	1536	1538	1536
q20	2070	1846	1859	1846
q21	5464	5244	5347	5244
q22	643	583	565	565
Total cold run time: 70554 ms
Total hot run time: 58107 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 190055 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit cc4607c658c00e6263e71cdfcb4e459c55cfa264, data reload: false

web_site	Doris	NULL	NULL	0	0	0	NULL	NULL	NULL	NULL	2023-12-26 22:43:58	2023-12-26 22:44:01	NULL	utf-8	NULL	NULL	
============================================
query1	1016	449	420	420
query2	6506	2828	2951	2828
query3	6713	224	220	220
query4	31344	23390	23504	23390
query5	4394	487	462	462
query6	290	229	194	194
query7	4625	318	299	299
query8	308	246	235	235
query9	9498	2684	2685	2684
query10	481	248	265	248
query11	17962	15132	15220	15132
query12	162	99	105	99
query13	1639	427	412	412
query14	11122	7615	7821	7615
query15	324	178	200	178
query16	7673	445	456	445
query17	query18	1584	308	342	308
query19	360	170	157	157
query20	121	120	119	119
query21	211	110	114	110
query22	5467	4941	4800	4800
query23	34503	33998	33798	33798
query24	12132	2543	2551	2543
query25	842	421	406	406
query26	2080	154	151	151
query27	2569	334	332	332
query28	query29	924	437	419	419
query30	301	157	155	155
query31	1079	821	856	821
query32	97	59	53	53
query33	788	307	301	301
query34	1025	518	551	518
query35	890	766	755	755
query36	1124	944	968	944
query37	247	83	72	72
query38	4197	4079	4101	4079
query39	1487	1412	1456	1412
query40	285	100	99	99
query41	48	42	45	42
query42	118	103	99	99
query43	539	484	481	481
query44	1230	811	812	811
query45	186	165	165	165
query46	1208	721	714	714
query47	1955	1845	1876	1845
query48	429	342	324	324
query49	1281	410	420	410
query50	809	399	392	392
query51	7137	7070	7141	7070
query52	105	93	96	93
query53	263	182	190	182
query54	1300	421	418	418
query55	88	84	83	83
query56	275	302	239	239
query57	1346	1217	1263	1217
query58	264	242	221	221
query59	3317	2987	3012	2987
query60	269	291	259	259
query61	114	109	103	103
query62	876	675	689	675
query63	221	187	192	187
query64	6947	678	647	647
query65	3294	3234	3207	3207
query66	1308	299	302	299
query67	16068	15615	15644	15615
query68	5654	564	555	555
query69	445	251	249	249
query70	1190	1109	1131	1109
query71	404	250	254	250
query72	6650	4016	4091	4016
query73	780	367	355	355
query74	10378	9025	9047	9025
query75	3574	2680	2699	2680
query76	3248	1097	1135	1097
query77	551	286	279	279
query78	9979	9483	9759	9483
query79	1124	597	600	597
query80	775	420	421	420
query81	484	224	224	224
query82	978	121	114	114
query83	283	145	143	143
query84	288	68	70	68
query85	1096	309	303	303
query86	317	312	284	284
query87	4737	4533	4468	4468
query88	3532	2229	2162	2162
query89	408	298	303	298
query90	2193	189	192	189
query91	141	107	105	105
query92	64	51	54	51
query93	1081	550	551	550
query94	829	286	257	257
query95	352	254	251	251
query96	612	291	285	285
query97	2856	2722	2708	2708
query98	214	191	194	191
query99	1554	1350	1291	1291
Total cold run time: 294136 ms
Total hot run time: 190055 ms

@924060929
Copy link
Contributor Author

run buildall

@924060929
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 39872 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 28a91e434526c7ff3c9d603310e1b8b09ea2f215, data reload: false

------ Round 1 ----------------------------------
q1	17593	7412	7230	7230
q2	2054	197	182	182
q3	10599	1093	1158	1093
q4	10572	790	708	708
q5	7607	2716	2689	2689
q6	240	147	145	145
q7	977	654	611	611
q8	9250	1917	1903	1903
q9	6663	6446	6377	6377
q10	7010	2305	2337	2305
q11	474	257	260	257
q12	435	227	237	227
q13	17803	2963	2962	2962
q14	260	216	222	216
q15	556	498	492	492
q16	644	593	582	582
q17	985	581	522	522
q18	7221	6707	6631	6631
q19	1451	1043	1028	1028
q20	468	184	182	182
q21	4097	3222	3214	3214
q22	377	318	316	316
Total cold run time: 107336 ms
Total hot run time: 39872 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7289	7249	7249	7249
q2	334	230	232	230
q3	2930	2741	2963	2741
q4	2039	1840	1790	1790
q5	5638	5772	5693	5693
q6	227	146	142	142
q7	2298	1809	1871	1809
q8	3419	3566	3550	3550
q9	8940	9003	9284	9003
q10	3613	3533	3533	3533
q11	622	503	514	503
q12	835	588	622	588
q13	13603	3162	3141	3141
q14	334	281	272	272
q15	566	505	542	505
q16	693	642	634	634
q17	1871	1647	1636	1636
q18	8333	7618	7724	7618
q19	3461	1516	1576	1516
q20	2061	1851	1890	1851
q21	5848	5446	5496	5446
q22	639	592	564	564
Total cold run time: 75593 ms
Total hot run time: 60014 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 199138 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 28a91e434526c7ff3c9d603310e1b8b09ea2f215, data reload: false

query1	1332	962	966	962
query2	6254	2441	2236	2236
query3	11046	4812	4845	4812
query4	33414	23418	23477	23418
query5	4170	490	476	476
query6	298	196	203	196
query7	4001	310	317	310
query8	297	233	240	233
query9	9588	2701	2705	2701
query10	469	256	259	256
query11	18055	15215	15328	15215
query12	155	104	110	104
query13	1587	432	414	414
query14	9248	8137	7742	7742
query15	275	201	195	195
query16	8273	477	473	473
query17	1622	646	610	610
query18	2183	348	346	346
query19	386	177	167	167
query20	124	114	122	114
query21	219	111	117	111
query22	4713	4261	4619	4261
query23	34949	33985	33979	33979
query24	10719	2675	2657	2657
query25	679	409	416	409
query26	1236	160	161	160
query27	2418	355	346	346
query28	7811	2424	2428	2424
query29	853	431	431	431
query30	231	159	152	152
query31	1043	863	855	855
query32	96	60	57	57
query33	769	318	323	318
query34	1005	543	534	534
query35	948	786	755	755
query36	1112	973	977	973
query37	138	80	75	75
query38	4375	4233	4175	4175
query39	1530	1467	1477	1467
query40	228	106	101	101
query41	45	42	48	42
query42	116	115	114	114
query43	537	501	499	499
query44	1399	878	830	830
query45	198	172	176	172
query46	1216	784	755	755
query47	2034	1920	1972	1920
query48	430	336	335	335
query49	933	400	391	391
query50	859	409	427	409
query51	7270	7320	7274	7274
query52	113	109	96	96
query53	283	192	195	192
query54	1089	428	430	428
query55	85	82	81	81
query56	284	259	258	258
query57	1288	1169	1173	1169
query58	233	236	238	236
query59	3199	3160	3244	3160
query60	295	270	282	270
query61	123	131	139	131
query62	876	693	716	693
query63	227	196	204	196
query64	4070	726	727	726
query65	3416	3292	3330	3292
query66	767	315	318	315
query67	16625	15537	15574	15537
query68	7113	560	547	547
query69	523	263	257	257
query70	1207	1096	1155	1096
query71	518	270	262	262
query72	6644	4130	4036	4036
query73	839	365	380	365
query74	9925	8844	8896	8844
query75	4194	2642	2678	2642
query76	4500	1136	1053	1053
query77	674	344	282	282
query78	10355	11635	9990	9990
query79	1182	626	613	613
query80	745	452	457	452
query81	510	252	251	251
query82	281	127	120	120
query83	203	143	146	143
query84	282	72	138	72
query85	770	320	320	320
query86	369	313	296	296
query87	4614	4551	4325	4325
query88	3840	2259	2237	2237
query89	426	295	292	292
query90	2162	195	189	189
query91	147	103	107	103
query92	72	54	53	53
query93	1095	550	542	542
query94	820	285	292	285
query95	362	260	250	250
query96	637	284	283	283
query97	2865	2699	2695	2695
query98	227	197	193	193
query99	1543	1330	1309	1309
Total cold run time: 305503 ms
Total hot run time: 199138 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.92 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 28a91e434526c7ff3c9d603310e1b8b09ea2f215, data reload: false

query1	0.04	0.04	0.03
query2	0.07	0.05	0.03
query3	0.23	0.06	0.06
query4	1.63	0.10	0.11
query5	0.43	0.41	0.41
query6	1.19	0.65	0.65
query7	0.02	0.01	0.01
query8	0.04	0.03	0.03
query9	0.60	0.51	0.49
query10	0.56	0.56	0.56
query11	0.15	0.11	0.11
query12	0.15	0.11	0.10
query13	0.60	0.59	0.60
query14	2.77	2.76	2.85
query15	0.90	0.84	0.83
query16	0.39	0.37	0.38
query17	1.05	1.05	1.06
query18	0.24	0.22	0.21
query19	1.86	1.82	1.99
query20	0.01	0.02	0.01
query21	15.36	0.60	0.59
query22	3.06	2.27	1.03
query23	16.97	0.86	0.84
query24	3.28	1.31	0.90
query25	0.12	0.22	0.07
query26	0.59	0.14	0.13
query27	0.06	0.05	0.04
query28	10.74	1.11	1.09
query29	12.58	3.27	3.21
query30	0.25	0.06	0.06
query31	2.87	0.39	0.39
query32	3.25	0.47	0.46
query33	3.06	3.10	3.17
query34	16.99	4.46	4.52
query35	4.55	4.52	4.52
query36	0.70	0.50	0.49
query37	0.09	0.06	0.06
query38	0.05	0.04	0.03
query39	0.04	0.03	0.03
query40	0.17	0.13	0.12
query41	0.08	0.03	0.03
query42	0.04	0.02	0.02
query43	0.04	0.03	0.03
Total cold run time: 107.87 s
Total hot run time: 31.92 s

yiguolei pushed a commit that referenced this pull request Dec 22, 2024
@924060929
[opt](nereids) optimize rewrite of synchronize materialize view (#45748…
bb543b8 
…) (#45752)

cherry pick from #45748
@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Dec 23, 2024
@github-actions GitHub Actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions GitHub Actions
Copy link
Contributor

PR approved by anyone and no changes requested.

@924060929 924060929 merged commit e7d2fed into apache:master Dec 23, 2024
27 of 28 checks passed
924060929 added a commit to 924060929/incubator-doris that referenced this pull request Dec 23, 2024
@924060929
[opt](nereids) optimize rewrite of synchronize materialize view (apac…
3bbe497 
…he#45748)

optimize rewrite of synchronize materialize view
1. cache toSql
2. fast parse UnboundSlot in NereidsParser.parseExpression

(cherry picked from commit e7d2fed)
@924060929 924060929 mentioned this pull request Dec 23, 2024
Merged
morrySnow pushed a commit that referenced this pull request Dec 24, 2024
@924060929
[opt](nereids) optimize rewrite of synchronize materialize view (#45748…
4fb8403 
…) (#45816)

cherry pick from #45748
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@morrySnow morrySnow morrySnow approved these changes

@starocean999 starocean999 starocean999 approved these changes

Assignees
No one assigned
Labels
approved Indicates a PR has been approved by one committer. dev/2.1.8-merged dev/3.0.4-merged reviewed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
6 participants
@924060929 @hello-stephen @doris-robot @starocean999 @morrySnow @yiguolei